On April 6, 2023, the New York City Department of Consumer and Worker Protection ("DCWP") issued a Notice of Adoption of Final Rule to implement Local Law 144 of 2021, legislation regarding automated employment decision tools ("AEDT Law"). DCWP also announced that it will begin enforcement of the AEDT Law and Final Rule on July 5, 2023. Pursuant to the AEDT Law, an employer or employment agency that uses an automated employment decision tool ("AEDT") in NYC to screen a candidate or employee for an employment decision must subject the tool to a bias audit within one year of the tool's use, make information about the bias audit publicly available, and provide notice of the use of the tool to employees or job candidates. The Final Rule, which follows the DCWP's previous proposals in September and December 2022 and a review of public comments, aims to: The Final Rule clarifies certain phrases within the AEDT Law's definition "Automated Employment Decision Tool." First, "to substantially assist or replace discretionary decision making" means: (1) "to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered"; (2) "to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set"; or (3) "to use a simplified output to overrule conclusions derived from other factors including human decision-making."

Before we can discuss how this law applies, let's start with what the law actually says. This is a fairly broad bill, and one that requires a bit of unpacking before understanding the responsibilities of the employers. Unfortunately, the law, as written, does not give much guidance for employers. Let's attempt to unpack what the bill does provide. Before discussing what this means for employers, let's look at some definitions: The Bias Audit must include testing of the AI Tool to assess the tool's disparate impact on persons of any Component 1 Category.

No. 1894-A, specifically regulates the use of "automated employment decision tools" in making employment decisions, including "any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons." The law protects candidates and employees interviewing and working in New York City and provides that an automated employment decision tool may not be used to screen such candidates for employment and promotion unless the tool: (i) has been subject to a "bias audit" conducted no more than one year prior to the use of such tool; and (ii) a summary of the results of the most recent bias audit of such tool, as well as the distribution date of the tool, have been made publicly available on the website of the employer or employment agency prior to the use of such tool. A bias audit is an "an impartial evaluation by an independent auditor," and includes, without limitation, "the testing of an automated employment decision tool to assess the tool's disparate impact on persons of any [gender, race and job level] required to be reported by employers [on the Employer Information Report EEO-1] pursuant to [federal law]." Notably, the law does not state who or what qualifies as an "independent auditor." The law also requires that the New York City employer or employment agency satisfy certain notice requirements.

The California Fair Employment and Housing Council (FEHC) recently took a major step towards regulating the use of artificial intelligence (AI) and machine learning (ML) in connection with employment decision-making. On March 15, 2022, the FEHC published Draft Modifications to Employment Regulations Regarding Automated-Decision Systems, which specifically incorporate the use of "automated-decision systems" in existing rules regulating employment and hiring practices in California. The draft regulations seek to make unlawful the use of automated-decision systems that "screen out or tend to screen out" applicants or employees (or classes of applicants or employees) on the basis of a protected characteristic, unless shown to be job-related and consistent with business necessity. The draft regulations also contain significant and burdensome recordkeeping requirements. Before the proposed regulations take effect, they will be subject to a 45-day public comment period (which has not yet commenced) before FEHC can move toward a final rulemaking.

On Nov. 10, 2021, the New York City Council passed a bill that regulates employers and employment agencies' use of "automated employment decision tools" in making employment decisions. The bill was returned without Mayor Bill de Blasio's signature and lapsed into law on Dec. 11, 2021. The new law takes effect on Jan. 1, 2023. This new law is part of a growing trend towards examining and regulating the use of artificial intelligence (AI) in hiring, promotional and other employment decisions. The new law regulates employers and employment agencies' use of "automated employment decision tools" on candidates and employees residing in New York City.

On November 10, 2021, the New York City Council passed a bill prohibiting employers and employment agencies from using automated employment decision tools to screen candidates or employees, unless a bias audit has been conducted prior to deploying the tool (the "Bill"). The Bill defines an "automated employment decision tool" as any computational process (either derived from machine learning, statistical modeling, data analytics, or artificial intelligence) that issues a simplified output (e.g., a score, classification or recommendation) to substantially assist or replace human decision-making for employment decisions that have an impact natural persons. Under the Bill, use of such a tool is permissible if it has been the subject of a bias audit (i.e., an impartial evaluation by an independent auditor, conducted no more than one year prior to the use of the tool) and a summary of the audit results are made publicly available on the website of the employer or employment agency before deployment of the tool. Moreover, employers or employment agencies are required to provide notice to employees or candidates residing in NYC that an automated employment decision tool will be used to assess or evaluate their candidacy, no less than ten business days before using the tool. Candidates also have the right to request an alternative selection process or accommodation under the Bill.

New York City will become the latest jurisdiction to regulate an employer's use of artificial intelligence (AI) and other "automated employment decision tools" in screening job candidates. The law is intended to curb bias in hiring and promotion decisions. Effective January 2, 2023, employers and employment agencies will be prohibited from using automated decision tools for screening employment or promotion candidates unless: (1) the tool has undergone an independent bias audit no more than one year prior to its use; and (2) certain information relating to the audit results is made publicly available on the employer's or employment agency's website. Additionally, companies will be required to notify employees or job applicants whether an AI tool was used to make employment decisions (amongst other notice requirements). NYC's law comes on the heels of the Equal Employment Opportunity Commission's (EEOC) announcement in October 2021 of an initiative to examine how AI technology is "fundamentally changing" the way employment decisions are made.

Joining Illinois and Maryland, on November 10, 2021, the New York City Council approved a measure, Int. The Bill, which is awaiting Mayor DeBlasio's signature, is to take effect on January 1, 2023. Should the Mayor not sign the Bill within thirty days of the Council's approval (i.e., by December 10), absent veto, it will become law. The Bill defines "automated employment decision tool" as "any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence," which scores, classifies, or otherwise makes a recommendation, that is used to substantially assist or replace the decision-making process from that of an individual. The Bill exempts automated tools that do not materially impact individuals, such as a junk email filter, firewall, calculator, spreadsheet, database, data set, or other compilation of data.

Big Data analytics supported by AI algorithms can support skills localization and retrieval in the context of a labor market intelligence problem. We formulate and solve this problem through specific DataOps models, blending data sources from administrative and technical partners in several countries into cooperation, creating shared knowledge to support policy and decision-making. We then focus on the critical task of skills extraction from resumes and vacancies featuring state-of-the-art machine learning models. We showcase preliminary results with applied machine learning on real data from the employment agencies of the Netherlands and the Flemish region in Belgium. The final goal is to match these skills to standard ontologies of skills, jobs and occupations.